Content prediction based on pixel-based vectors

ABSTRACT

Methods and systems disclosed herein relate generally to systems and methods for predicting content based on vector data structures generated from image pixels. A content-prediction application accesses a color palette having two or more color-palette categories and selects a first color of the color palette. The content-prediction application generates a first vector based on a set of pixel values that represent the first color of the color palette. The content-prediction application determines a distance metric between the first vector and a second vector, in which the second vector is identified by applying a convolutional neural network model on an image depicting an item that includes a second color. In response to determining that the distance metric is less than a predetermined threshold, the content-prediction application selects the content item corresponding to the second vector.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/915,328, filed on Jun. 29, 2020, now allowed, the contents of all ofwhich are incorporated herein by reference.

TECHNICAL FIELD

This disclosure generally relates to methods that facilitate predictionof content based on vector data structures generated from image pixels.More specifically, but not by way of limitation, this disclosure relatesto applying machine-learning models to pixels of an image to generatecoordinate values of the vectors for prediction of content.

BACKGROUND

Image-search systems generally involve processing a search querycontaining a plurality of words and performing a database searchoperation to identify a plurality of images having characteristics thatmatch one or more keywords of the search query. The search query mayinclude a natural text string, in which the image-search systems maysegment the text string into one or more keywords that identify thecharacteristics of the images for retrieval.

Reverse image search is a content-based image retrieval query techniquethat involves an image-search system receiving a sample image or ahyperlink to the sample image, extract one or more features, and performa database search based on the extracted features. By using theextracted features, reverse image search facilitates identification ofimages that include similar content as the sample image.

SUMMARY

According to certain embodiments, a content-prediction applicationaccesses a color palette having two or more color-palette categories andselects a first color of the color palette. The content-predictionapplication generates a first vector based on a set of pixel values thatrepresent the first color of the color palette. The content-predictionapplication determines a distance metric between the first vector and asecond vector, in which the second vector is identified by applying aconvolutional neural network model on an image depicting an item thatincludes a second color. The content-prediction application determinesthat the distance metric between the first and second vectors is lessthan a predetermined threshold, in which the predetermined thresholdindicates a degree of color similarity between the first vector andsecond vector. In response to determining that the distance metric isless than a predetermined threshold, the content-prediction applicationselects the content item corresponding to the second vector. Thecontent-prediction application generates, for output on a userinterface, an item catalog that includes the selected item.

These illustrative embodiments are mentioned not to limit or define thedisclosure, but to provide examples to aid understanding thereof.Additional embodiments are discussed in the Detailed Description, andfurther description is provided there.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Features, embodiments, and advantages of the present disclosure arebetter understood when the following Detailed Description is read withreference to the accompanying drawings.

FIG. 1 illustrates a computing environment for predicting content basedon pixel-based vectors in accordance with some embodiments.

FIG. 2 illustrates a process for predicting content based on pixel-basedvectors in accordance with some embodiments.

FIG. 3 illustrates a set of color palettes in accordance with someembodiments.

FIG. 4 illustrates a process for selecting color palettes based onlayout data extracted from previous interaction data in accordance withthe embodiments.

FIG. 5 illustrates a process for selecting color palettes based onapplying neural networks to previous interaction data in accordance withthe embodiments.

FIG. 6 illustrates a schematic diagram for using convolutional neuralnetworks to predict color signatures of content items in accordance withsome embodiments.

FIG. 7 depicts a computing system configured for predicting contentbased on pixel-based vectors in accordance with some embodiments.

DETAILED DESCRIPTION OF THE INVENTION

Conventional image-search systems are limited to identifying images thatinclude features that directly correspond to the keywords or featuresextracted from a search query (e.g., text string, images). For example,a search engine can identify an item having a color that matches thecolor specified in keywords previously submitted by the user. Forreverse image search techniques, only images that have the same featuresas the sample image can be retrieved. The conventional techniques thusdo not otherwise identify content that are different but still could berelevant to the image submitted by the user. In addition, it ischallenging to automatically identify data objects that satisfyparameters (e.g., images arranged in specific color proportions)inferred from the preference of the user. This includes identifyingcontent that includes colors arranged into specific proportions thatmatches the preference of the user.

Accordingly, prior database systems generally do not accommodateidentifying content from specific parameters without using complex setof search-syntax commands. Thus, prior techniques are typically timeconsuming, inaccurate, and inefficient with respect to identifying colorinformation that would accommodate specific color usage patterns andpreferences of a user. Moreover, inferring patterns and proportions ofdifferent colors for content search can be difficult because apreference of a user can vary over time.

Certain embodiments described herein can address these problems byproviding prediction of content based on vector data structuresgenerated from image pixels. A content-prediction application accesses acolor palette that includes colors from two or more color-palettecategories. In some instances, the colors of the color palette arepredicted based on interaction data associated with a user. The colorpalette may correspond to a particular color scheme specified by a user.For example, a user accesses a website that provides advertising forclothing items. The content-prediction application identifies a colorpalette that matches the color preference of the user. The colorpreference is determined by identifying a set of colors derived fromhistorical activity of the user. In this example, the color preferencecorresponds to a warm-color scheme.

After the color palette has been identified, the content-predictionapplication converts one or more colors of the color palette into a setof color-palette vectors for a color-embedding space. Specifically, thecontent-prediction application selects a first color from acolor-palette category of the color palette. A set of pixel values thatcharacterize the first color can be identified and generate a firstvector (hereinafter “color-palette vector”) that represents the firstcolor at a first location within a color-embedding space. In someembodiments, the color-embedding space corresponds to a statisticalspace in which two vectors can be compared to determine a degree ofsimilarity. The process of converting the color to a correspondingcolor-palette vector can be repeated for other colors of the colorpalette, such that the color-embedding space includes the set ofcolor-palette vectors that correspond to the colors of the colorpalette. In some instances, the content-prediction application applies aweight parameter to one or more color-palette vectors of the colorpalette based on their respective color-palette categories.

The content-prediction application identifies content vectors that arewithin a predetermined distance (e.g., a distance metric indicatingnearest neighbor) from each of the color-palette vectors that representthe colors of the selected color palette. In some instances, thecontent-prediction application identifies content vectors for eachcolor-palette vector of the color palette, such that content items thatmatch at least one color of the color palette can be identified. In someembodiments, each content vector represents a content item that isassociated with a particular content type and includes colors that matchone or more colors of the color palette. The content-predictionapplication can generate each of the content vectors by accessing pixelvalues of each image representing the content, then applying aconvolutional neural network to derive values that represent a locationof the content vector within the color-embedding space. The contentvector can be further applied with a weight parameter, based on a typeof color-palette category that is applicable to the content item.Referring to the above example, a content vector in the color-embeddingspace may be an image that represents a clothing item. To compare thecontent vector and the color-palette vector, a distance metriccorresponding to a degree of similarity between the two vectors can bedetermined. In some instances, the distance metric is an Euclideandistance or a cosine similarity metric between the two vectors.

The content-prediction application selects at least one content vectorif the distance metric calculated between the vector and thecolor-palette vector is below a predetermined threshold. A content itemcorresponding to the at least one content vector can be identified andpresented to a user, in which the content includes one or more colorsthat match the colors of the color palette. In some instances, thecontent items are aggregated into a catalog (for example) and presentedto a user interface. Referring to the above example, the clothing itemsthat are represented by the identified content vectors can be accessedand presented on a first page of a website, at which the user can viewand purchase the recommended clothing items.

Accordingly, one or more colors of the color palette can be encoded asvectors in the color-embedding space. The vectors can be compared withother vectors in color-embedding space that represent content, e.g.,items in an item catalog. A set of vectors located within apredetermined distance of those of the colors can be identified. A setof content objects that are represented by the respective set of vectorscan be retrieved, at which the content objects can be recommended to theuser. In some instances, a user interface is modified such that userinterface elements depicting the content objects can be arranged fordisplay at a more visible portion of the user interface.

In some instances, one or more colors of the color palettes are selectedby a user via a user interface. The selection of the color palettes canbe based on a particular color scheme, such as cool, warm, and bright.In some instances, the color palettes are selected based on a non-colorcategory, such as a season or weather category (e.g., spring), alocation category (e.g., beach), and an event category (e.g., New Year'sDay). For example, the user interface displays a number of preconfiguredcolor palettes, at which the user may select one of the color palettesbased on her or his preference. Each color palette may include a maincolor-palette category that corresponds to the highest proportion ofcolors in the color palette, a neutral color-palette category thatcorresponds to a lower proportion of colors relative to the maincolor-palette category, and an accent color-palette category thatcorresponds to the lowest proportion of colors relative to the maincolor-palette category and a neutral color-palette category. The userinterface may present the color palette based on the proportion of itscolors. In some instances, the user interface displays a number of colorschemes and presents a set of color palettes in response to a selectionof a particular color scheme. The user may select a color palette fromthe set, such that content items that match one or more colors of thecolor palette can be identified.

In some instances, a type of item is inferred based on the color-palettecategory of the color palette. For example, a first type of item (e.g.,a sofa) can be selected by correlating the first type with the maincolor-palette category, a second type of item (e.g., a rug) can beselected by correlating the second type with the neutral color-palettecategory, and a third type of item (e.g., a lamp) can be selected bycorrelating the third type with the accent color-palette category. Thetype of item selected of a particular color-palette category can beinferred based on a size of the item, a user query submitted in an itemcatalog database, and the like. Additionally or alternatively,user-interface elements representing the content items can be relocatedbased on a selection of a particular color palette. For example, a webpage that displays the item catalog can automatically relocatepresentation of user-interface elements, such that a first part of theweb page depicts content items having a color that is associated withthe main color-palette category, a second part of the web page depictscontent items having another color that is associated with the neutralcolor-palette category, and so forth. In some instances, the userinterface displays a number of content items that are proportional tothe type of the color-palette category. For example, the user interfacedisplays a large number of content items that are associated with themain color-palette category, while displaying a smaller number ofcontent items that are associated with the neutral color-palettecategory.

As a result, the user interface can display content items thatcorrespond to color palette preference of the user. The user may“refresh” the user interface to rearrange and/or reselect the contentitems that are based on the same or different color palette. Forexample, the user interface rearranges the content items within theresults in response to a user action, if the user does not find aproduct she or he is interested in, but still prefers to remain withinthe same color-palette (or section of the palette) and exploreadditional content items.

In some instances, the content-item prediction is implemented inaddition to an existing recommendation system. For example, a colorpalette can be selected using collaborative filtering techniques, inwhich data used to indicate the color-palette preference can be based onhistorical data accessed from other users.

Certain embodiments described herein improve existing database systemsby predicting data objects based on vector data structures derived fromimage pixels. Processing a plurality of image pixels to generate avector in an embedding space can facilitate accurate prediction of dataobjects that substantially match various color parameters in a searchquery. By utilizing color palettes as a search query instead of textstrings, various combinations of search parameters can be configuredwithout requiring complex syntax rules, such as specifying a certainpercentage of pixels to be present in an image-search result. Aparticular type of a content item can be inferred based on a colorcategory associated with the color palette, which cannot be implementedbased on reverse image search which requires exact matching of extractedfeatures. Further, prediction of data objects by converting a pluralityof images into vectors in the embedding space not only improvestransmission rate of data retrieval, but also increases an efficient useof computing resources to store and process images in networkenvironments. As a result of the above, an improvement of databasesystems can be achieved based on one or more implementations ofpredicting content based on vector data structures generated from imagepixels.

In addition to improvement of database systems, automatic relocation ofuser-interface elements (e.g., icons, interactive images, hyperlinks)based on the prediction of data objects as described in one or moreembodiments improves existing user-interface systems. The relocation ofa user-interface element based on a corresponding color category of thecolor palette to another portion of the user interface improvesaccessibility and efficiency of retrieving content. Accordingly, animprovement of user-interface systems can be achieved based on automaticrelocation of user-interface elements based on prediction of contentitems in response to a color-palette selection.

Computing Environment

FIG. 1 illustrates a computing environment 100 for predicting contentbased on pixel-based vectors in accordance with some embodiments. Theexemplary computing systems include a server system 102 and computingsystems 103 a and/or 103 b in communication via a network 106.

The server system 102 comprises a computer-readable medium such as aprocessor 104 that is communicatively coupled to a memory 108 and thatexecutes computer-executable program instructions and/or accessesinformation stored in the memory 108. The processor 104 may comprise amicroprocessor, an application-specific integrated circuit (“ASIC”), astate machine, or other suitable processing device. The processor 104can include any of a number of computer processing devices, includingone. Such a processor can include or may be in communication with acomputer-readable medium storing instructions that, when executed by theprocessor 104, cause the processor to perform the steps describedherein.

The server system 102 can provide access to the color-palette content114. The color-palette content 114 may be stored in any suitablecomputer-readable medium and execute on any suitable processor. In someembodiments, the color-palette content 114 is stored in the memory 108at the server system 102. The color-palette content 114 can be accessedby the server system 102 from a remote location via a network 106 andprovided to computing systems 103 a and 103 b. The color-palette content114 may include a plurality of color palettes that can be transmitted bythe server system 102 to the computing systems 103 a and 103 b. In someembodiments, the color-palette content 114 is generated by the serversystem 102 or by another system (e.g., application servers). Based onthe selection of the color palette, the server system 102 may predictcontent that include colors matching one or more colors of the selectedcolor palette.

A content-prediction application 115 stored in the memory 108 canconfigure the processor 104 to manage and provide access to thecolor-palette content 114. The content-prediction application 115 canconfigure the processor 104 to access some or all of the color-palettecontent 114 stored in the memory 108 or any other suitablecomputer-readable medium. The content-prediction application 115 mayprovide a social media service, a cloud service, or other networkservice providing access to the color palettes as well as identifying aset of items that match the color palette. A cloud service can include acollection of computing resources, including computing systems and/orapplications, that can be provided as an online service via a datanetwork. The collection of computing systems and/or hardware can berepresented as a single service. The cloud service can provide a digitalhub for browsing, creating, sharing, and otherwise using electroniccontent using one or more applications provided via the cloud service.

In some instances, the content-prediction application 115 accesses andtransmits color palettes in response to one or more selections performedby a user (e.g., the user of the computing system 103 a). Thecontent-prediction application 115 may also predict the color palettesbased on previous interaction data associated with the user. Forexample, the interaction data associated with the user are retrievedfrom browsing data stored in computing system 103 a. The color palettemay correspond to a particular color scheme, including bright-colorscheme and cool-color scheme.

The content-prediction application 115 may access the color paletteselected by the content-prediction application 115 and use the colorpalette to predict content for the user. The content-predictionapplication 115 identifies pixel values that characterize each color ofthe color palette and generate a color-palette vector based on the pixelvalues. The color-palette vector include values that identify a firstlocation within a color-embedding space. The content-predictionapplication 115 may calculate a distance metric between a color-palettevector with a content vector associated with the color-embedding space.Based on the distance metric, the content-prediction application 115 canpredict content that matches one or more colors of the color palette.

The server system 102 may also comprise a number of external or internaldevices such as input or output devices. For example, the server system102 is shown with an input/output (“I/O”) interface 112. A bus 110 canalso be included in the server system 102. The bus 110 cancommunicatively couple one or more components of the server system 102.

FIG. 1 also illustrates the content-prediction application 115 comprisedin the memory 108 of the server system 102. The content-predictionapplication 115 can include one or more modules. The content-predictionapplication 115 can configure the server system 102 to access, modify,transmit, or otherwise use the color-palette content 114 in response toinput provided by the client applications 128 a and/or 128 b. Forexample, the client application 128 a transmits a query to retrieve acolor palette of the color-palette content 114, at which the serversystem 102 may access the color palette by processing the query througha database system.

The content-prediction application 115 can also configure the serversystem 102 to allow multiple computing systems 103 a and/or 103 b orother client systems to access or otherwise use some or all of thecolor-palette content 114. In some embodiments, the content-predictionapplication 115 stores data in the memory 108 identifying clientaccounts via which the client applications 128 a and/or 128 b can accessthe content-prediction application 115. In other embodiments, thecontent-prediction application 115 accesses data from a remote datasource that identifies client accounts via which the client applications128 a and/or 128 b can access the content-prediction application 115.The content-prediction application 115 can store data describingassociations between client accounts and items of the color-palettecontent 114.

The server system 102 can include any suitable computing system forhosting the content-prediction application 115. In one embodiment, theserver system 102 may be a single computing system. In anotherembodiment, the server system 102 may be a virtual server implementedusing a number of computing systems connected in a grid or cloudcomputing topology.

Each of the computing systems 103 a and/or 103 b comprises acomputer-readable medium such as processors 118 a and/or 118 b,respectively. Each of the processors 118 a and/or 118 b iscommunicatively coupled to a memory 120 a and/or 120 b, respectively.Each of the processors 118 a and/or 118 b respectively executescomputer-executable program instructions and/or accesses informationstored in the memory 120 a and/or 120 b. Each of the processors 118 aand/or 118 b may comprise a microprocessor, an ASIC, a state machine, orother processor. Each of the processors 118 a and/or 118 b can includeany of a number of computer processing devices, including one. Such aprocessor can include or may be in communication with acomputer-readable medium. The computer-readable medium storesinstructions that when executed by the processor, cause the processor toperform the steps described herein.

The computing systems 103 a and/or 103 b may also comprise a number ofexternal or internal devices such as a mouse, a CD-ROM, DVD, a keyboard,a display, audio speakers, one or more microphones, or any other inputor output devices. For example, each of the computing systems 103 aand/or 103 b is respectively shown with input/output (“I/O”) interfaces124 a, 124 b and display devices 126 a, 126 b. Buses 122 a, 122 b can berespectively included in the computing systems 103 a and/or 103 b. Eachof the buses 122 a, 122 b can communicatively couple one or morecomponents of the computing systems 103 a and/or 103 b.

FIG. 1 also illustrates the client applications 128 a and/or 128 brespectively comprised in the memory 120 a and/or 120 b of the computingsystems 103 a and/or 103 b. Each of the client applications 128 a and/or128 b can include one or more software modules. The client applications128 a and/or 128 b can respectively configure the processors 118 aand/or 118 b to modify or otherwise access the color-palette content 114provided by the server system 102 via the network 106. The clientapplications 128 a and/or 128 b can respectively configure theprocessors 118 a and/or 118 b to respectively render some or all of thecolor-palette content 114 and predicted content generated by thecontent-prediction application 115 for display at their respectivedisplay devices. For example, each of the client applications 128 aand/or 128 b is an internet browser application, a social mediaapplication, and the like. A network session for accessing or otherwisemodifying the color-palette content 114 can be established by the clientapplications 128 a and/or 128 b via the network 106 between computingsystems 103 a and/or 103 b and the server system 102.

In some embodiments, the computing systems 103 a and/or 103 b includeany suitable client devices for communicating via a network 106 andexecuting the client applications 128 a and/or 128 b. Non-limitingexamples of a computing device include a desktop computer, a tabletcomputer, a smart phone, or any other computing device suitable forusing electronic content. In other embodiments, the computing systems103 a and/or 103 b include server systems for providing electroniccontent items via the client applications 128 a and/or 128 b.

Process for Predicting Content Based on Color-Palette Preference

FIG. 2 illustrates a process 200 for predicting content based onpixel-based vectors in accordance with some embodiments. Forillustrative purposes, the process 200 is described with reference tothe components illustrated in FIG. 1 , though other implementations arepossible. For example, the program code for content-predictionapplication 115 of FIG. 1 , which is stored in a non-transitorycomputer-readable medium, is executed by one or more processing devicesto cause the server system 102 to perform one or more operationsdescribed herein.

At step 202, a content-prediction application (e.g., thecontent-prediction application 115 of FIG. 1 ) accesses a color palettethat includes multiple color-palette categories. The color paletteincludes multiple colors that can be used to predict content. The colorpalette may include three different color-palette categories, includinga main color-palette category, a neutral color-palette category, and anaccent color-palette category. In some instances, the content-predictionapplication receives previous interaction data indicative of historicaluser activity that occurred via a browser application. The previousinteraction data may include color-layout data of a web page and/or oneor more image objects to which at least some of the historical useractivity was directed. In the case of image objects, thecontent-prediction application may extract pixel values of each imageobject, such that a machine-learning model may identify multiple colors(e.g., two or more colors) used to identify a color-palette preferenceof the user.

In some instances, the content-prediction application applies amachine-learning model, such as a neural network, to the previousinteraction data to infer the color-palette preference of the userassociated with the previous interaction data. For example, eachaccessed web page indicated by the previous interaction data isprocessed to generate a vector in an embedding space, in which theplotted vectors can be used to infer the color-palette preference of theuser. The content-prediction application determines and accesses thecolor palette having colors that match the inferred color-palettepreference.

In some instances, the color palette is selected through one or moreuser-interface actions performed by a user. For example, a set of colorpalettes are presented to the user. The set of color palettes may becategorized into different color schemes, including a warm-color scheme,a cool-color scheme, and a bright-color scheme. In some instances, thecolor palettes are selected based on a non-color category, such as aseason or weather category (e.g., spring, winter), a location category(e.g., beach, city), and an event category (e.g., New Year's Day,birthday). The user can select a color palette from the set of colorpalettes through a user action performed via the user interface. In someinstances, the set of color palettes are presented as a multi-facetedsearch, in which the user interface can generate a first set of itemswhen the user performs a selection of a particular color palette on theuser interface. The first set of items can be added or replaced with asecond set of items when the user performs another selection of anothercolor palette on the user interface. In some instances, the user selectsmultiple color palettes which the system identifies sets of items thatcorrespond to one or more color of the multiple color palettes.

At step 204, the content-prediction application identifies a set ofpixel values that characterize a color of the color palette. The colorcan be represented by the set of pixel values across different colorchannels and may formatted in different color models, including RGB,LMS, and CMYK models. In some instances, the color identified by the setof pixel values is selected based on its color-palette category. Forexample, the set of pixel values are identified for a color that isassociated with a main color-palette category. Additionally oralternatively, multiple sets of pixel values can be identified forrespective colors in the color palette.

At step 206, the content-prediction application generates acolor-palette vector based on the set of pixel values, in which thecolor-palette vector represents the color at a first location within acolor-embedding space. In some instances, a trained neural network modelis applied to the set of pixel values to derive a set of graphcoordinates for identifying the location of the color-palette vector atthe color-embedding space. In some instances, a color-palette vector isgenerated for other remaining colors of the color-palette, such that aset of multiple color-palette vectors represent the selected colorpalette. Additionally or alternatively, the color-palette vector isapplied with a weight parameter, in which its value depends on a type ofthe color-palette category (e.g., a main color-palette category)associated with the particular color.

At step 208, the content-prediction application identifies a contentvector specifying a second location within the color-embedding space.The content vector identifies a color signature of the a content itemretrieved from a database (e.g., an item-catalog database). In someinstances, the content-prediction application successively performs aconvolution, a non-linearity conversion (e.g., Rectified Linear Unit(ReLU)), a pooling or sub sampling, and a classification (FullyConnected Layer) operations, which form a single convolutional layer.The content-prediction application may apply a plurality ofconvolutional layers to an image of the content item to generate thecontent vector used to predict whether the content item includes colorsthat match the selected color palette. In some instances, thecontent-prediction application identifies multiple content vectors thatrepresent the same content item, in which the content item is identifiedto include multiple colors. Additionally or alternatively, the contentvector is applied with the weight parameter, in which its value dependson a type of the color-palette category (e.g., a main color-palettecategory) applicable to the content item. The value of the weightparameter can be identical to the value applied for the color-palettevector.

At step 210, the content-prediction application determines that adistance metric between the color-palette vector and the content vectoris less than a predetermined threshold. To determine that the distancemetric is less than the predetermined threshold, the distance metricbetween two vectors in the color-embedding space can be calculated, inorder to identify a degree of similarity between the color of the colorpalette and the color signature of the content item. In someembodiments, the predetermined threshold indicates a likelihood that thetwo colors that are substantially similar. The distance metric can be acosine distance, an Euclidean distance, Minkowski distance, and/orHamming distance. If the content item is represented by set of contentvectors, the content-prediction application calculates a distance metricbetween each of the set of content vectors and the nearest color-palettevector in the color embedding space, thereby generating a set ofdistance metrics. An average or a median value of the set of distancemetrics can be calculated, which is then identified whether such valueis less than the predetermined threshold.

At step 212, the content-prediction application selects a content itemthat includes a color that is represented by the content vector inresponse to determining that the distance metric being less than thepredetermined threshold. For example, a first clothing item (e.g., ajacket) that is represented by the content vector is identified. Inanother example, a second clothing item (e.g., a patterned sofa) isidentified based on determining that the average or median value of theset of distance metrics is below the predetermined threshold.

At step 214, the content-prediction application generates, for output ona user interface, an item catalog that includes the selected contentitem. In some instances, an item catalog is generated to include one ormore content items that include colors that substantially match one ormore colors associated with the color palette. The item catalog mayinclude a set of items in different categories. User-interface elements(e.g., icons, interactive images, hyperlinks) corresponding to theidentified items can be automatically relocated in the user interface ina manner that items regarded as closest to a particular color of thecolor palette can be placed on a visible portion of the user interface(e.g., a web page rendered by a browser application).

In some instances, the layout of user-interface elements is customizedsuch that content items can be presented on the user interface based onthe color-palette categories of the color palette. For example, a firstset of content items in the layout that correspond to a color of a maincolor-palette category are displayed in a more visible portion of theuser interface. A second set of content items in the layout thatcorrespond to another color of a neutral color-palette category can bedisplayed in a less visible portion of the user interface. In someinstances, the user interface displays a number of content items thatare proportional to the type of the color-palette category. For example,the user interface displays a large number of content items that areassociated with the main color-palette category, while displaying asmaller number of content items that are associated with the neutralcolor-palette category.

The steps in FIG. 2 can be repeated based on a user action on the userinterface. For example, the user may “refresh” the user interface torearrange and/or reselect the content items that are based on the sameor different color palette. In particular, the user interface rearrangesthe content items within the results in response to a user action, ifthe user does not find a product she or he is interested in, but stillprefers to remain within the same color-palette (or section of thepalette) and explore additional content items. Process 200 terminatesthereafter.

Example Color Palette

FIG. 3 illustrates a set of color palettes 300 in accordance with someembodiments. A set of color palettes can be accessed from a serversystem (e.g., the server system 102 of FIG. 1 ) and presented on a userinterface of a client application of a client system (e.g., the clientsystem 103A of FIG. 1 ). The set of color palettes 300 can be dividedinto multiple color schemes. The color scheme may group colors palettesin different color combinations to emit a particular visual effect. Forexample, the color scheme includes a warm-color scheme 305, a cool-colorscheme 310, a deep-color scheme 315, a light-color scheme 320, aneutral-color scheme 325, a colorful-color scheme 330, and so on. Insome instances, the user interface displays a number of color schemesand presents multiple color palettes in response to a selection of aparticular color scheme. The user may select a color palette thereafter,such that one or more items that match one or more colors of the colorpalette can be identified.

A color palette selected from a particular color scheme may include atleast two color-palette categories. Referring to FIG. 3 , a colorpalette 335 includes three color-palette categories: (i) a maincolor-palette category 340; (ii) a neutral color-palette category 345;(iii) and an accent color-palette category 350. The main color-palettecategory may include one or more colors that represent the primary colorof the color palette. In some instances, the primary color of the colorpalette is identified based on a color preference of the user. Theneutral color-palette category may include one or more colors thatsupport and balance the colors from the main color-palette category. Theaccent color-palette category may include one or more colors that addvariety to the colors of the main color-palette category and the neutralcolor-palette category.

In some embodiments, the color palette includes one or more rules thatspecifies the proportions of colors that should be depicted for aselected item. For example, the color palette includes a rule specifyingcolors from the main color-palette category should be at a higherproportion (60%) than colors from the neutral color-palette category(30%) and color from the accent color-palette category (10%). Oneordinarily skilled in the art can appreciate that other ratios of colorproportionality can be used. The user interface may present the colorpalette in a manner that reflects upon a proportion of colors in eachcolor-palette category.

After the color palette is selected, different types of items can beidentified based on colors associated with color-palette categories ofthe color palette. In particular, proportions of a color at a particularcolor-palette category can be used to determine a type of item. Forexample, a first clothing item (e.g., a jacket, pants) corresponds to acolor that is associated with the main color-palette category of thecolor palette. A second clothing item (e.g., hats) may correspond to acolor that is associated with the neutral color-palette category of thecolor palette. An accessory item (e.g., shoes) may correspond to a colorthat is associated with the accent color-palette category of the colorpalette. Based on the above example, a type of the identified items canbe based on a particular color-palette category.

In some instances, an item catalog is generated to include one or moreitems that include colors that substantially match one or more colorsassociated with the selected color palette. The item catalog may includea set of items in different categories. User-interface elements (e.g.,icons, interactive images, hyperlinks) corresponding to the identifieditems can be automatically relocated in the user interface in a mannerthat items regarded as closest to a color of the color palette can beplaced on the top of the catalog. In some instances, an item thatincludes a large proportion of color that matches a color of the maincolor-palette category is placed at a particular position of thegenerated item catalog. Another item that includes smaller proportion ofcolor that matches the color of the main color-palette category can beplaced at a lower position relative to the particular positioncorresponding to the item.

Selecting Color Palettes Based on Previous Interaction Data

Although the color palette is selected through one or moreuser-interface actions performed by a user, it can also be advantageousto select the color palettes based on a color-palette preferencepredicted for the user. The predicted color-palette preference can beidentified based on processing previous interaction data tracked for theuser (e.g., web pages accessed by the user). The color-palettepreference can be predicted by analyzing layout data extracted from theinteraction data or by applying a trained neural network model to theprevious interaction data of the user.

(a) Identifying Color-Palette Preference Based on Layout Data

FIG. 4 illustrates a process 400 for selecting color palettes based onlayout data extracted from previous interaction data in accordance withthe embodiments. For illustrative purposes, the process 400 is describedwith reference to the components illustrated in FIG. 1 , though otherimplementations are possible. For example, the program code forcontent-prediction application 115 of FIG. 1 , which is stored in anon-transitory computer-readable medium, is executed by one or moreprocessing devices to cause the server system 102 to perform one or moreoperations described herein.

At step 402, a content-prediction application (e.g., thecontent-prediction application 115 of FIG. 1 ) identifies color-inputdata from a web page of a set of web pages accessed by a user (e.g., theclient application 128 a of FIG. 1 ). The set of web pages representprevious interaction data which indicate historical user activity of theuser. For example, the content-prediction application accesses a URL ofa web page and identify image files based on their respective fileextensions (e.g., jpeg, jpg, png). The identification of the image filescan be facilitated based on a tag identifier of a document object model.Additionally or alternatively, the content-prediction application mayparse a portion of the web page to extract the layout data for the webpage, from which the color-input data can be identified. The portion canbe configured based on the tag identifier and/or a particular locationof the portion in the webpage. In some instances, the content-predictionapplication accesses the URL and retrieves cascading style sheet (CSS)files embedded in the URL.

At step 404. the content-prediction application determines a pluralityof colors from the color-input data. If the color-input data includeimage objects identified from the web page, the plurality of colors canbe identified based on pixels values (e.g., RGB value, color hex code)corresponding one or more pixels of the image objects. If thecolor-input data include CSS files of the web page, thecontent-prediction application can determine the plurality of colors byparsing the web page to identify content associated with particular tagproperties (e.g., style). The identified content for each tag can beprocessed to identify the color that is included in the web page.

At step 406, the content-prediction application aggregates theidentified colors (e.g., the color codes). In some instances, the colorcodes is stored in an array data structure, in which each index value ofthe array corresponds to the color code identified from the web page. Incase of image objects retrieved from the web page, pixels of each of theimage objects can be accessed to identify a set of colors that areaggregated for the web page. In some instances, each color of the set ofcolors of the image object is counted to identify a single color valuerepresentative of the image object (e.g., most frequently occurringcolor value in the image object).

At step 408, the content-prediction application processes the aggregatedcolors to identify a color representative of the web page. For example,the most frequently appearing color in the aggregated colors is selectedas the color representative of the web page. In some instances, the mostfrequently appearing color based on a subset of the aggregated colors(e.g., colors corresponding to the extracted image objects) is selectedas the color representative of the web page. In some instances, thepalette-selection repeats steps 402 to 408 to identify the colors thatrepresent the respective web pages of the set.

At step 410, the content-prediction application processes colorsrepresenting the set of web pages accessed by the user to determine acolor-palette preference for the user. For example, a set of colorsrepresenting the web pages are assigned as the color-palette preference,at which one or more color palettes can be identified based on thecolor-palette preference. In some instances, the color-palettepreference is used to identify one or more color schemes of the colorpalettes, such that the user interface of the client applicationpresents the color palettes corresponding to the identified colorschemes.

At step 412, the content-prediction application selects one or morecolor palettes based on the color-palette preference. For example, thecontent-prediction application identifies a color of the color-palettepreference and convert the color into a database query. Thecontent-prediction application can submit the query to a database in theserver system (e.g., the color-palette content 114 of FIG. 1 ) toretrieve a set of color palettes that include the color specified in thedatabase query. Process 400 terminates thereafter.

(b) Predicting Color-Palette Preference Based on Neural Networks

FIG. 5 illustrates a process 500 for selecting color palettes based onapplying neural networks to previous interaction data in accordance withthe embodiments. For illustrative purposes, the method 500 is describedwith reference to the components illustrated in FIG. 1 , though otherimplementations are possible. For example, the program code forcontent-prediction application 115 of FIG. 1 , which is stored in anon-transitory computer-readable medium, is executed by one or moreprocessing devices to cause the server system 102 to perform one or moreoperations described herein.

At step 502, a content-prediction application (e.g., thecontent-prediction application 115 of FIG. 1 ) accesses a set of webpages of the previous interaction data. At step 504, thecontent-prediction application applies a neural network (e.g., anartificial neural network) to the set of web pages to generate vectorsrepresentative of the respective web pages. In some instances, theneural network (e.g., a multilayer deep neural network) trained withwebsite data can be applied to a web page (e.g., a web page) of theprevious interaction data. For example, each word appearing in the webpage is assigned as an input value for a set of input values, and theinput values are applied to the neural network to generate anm-dimensional vector representative of the web page.

At step 506, the content-prediction application associates the vectorsin an embedding space. In some instances, web pages of the vectors thatshare common colors are located in close proximity to one another in theembedding space. For example, a high-dimensional, sparse vector includesa large set of values corresponding to the text data of the web page, inwhich each value of the set represents a separate English word of adictionary. The large set may include a million elements to represent amillion discrete words. Processing vectors with millions of elements maybe inefficient especially if computing resources of a given system islimited. To reduce the dimensionality of the vector, the neural networkmodel can process the text data in the web page such that a limiteddimension of values can be identified for the vector. In effect, wordsof the web page can be semantically encoded using as attributesidentified by the trained neural networks. The vector may thus capturethe color context of the web page.

Additionally or alternatively, other vectors derived from other users'previous interaction data can be plotted in the embedding space based oncollaborative filtering. The collaborative filtering can allowpredictions of the user's color-palette preference based on thehistorical user activity of other users. For example, each of the webpages accessed by various users is applied with the neural-networkmachine-learning model to generate a set of values, then the set ofvalues for each web page can embedded in the same embedding space as theembedding space that includes the vectors that represent web pagesaccessed by the user. This allows the color-palette preference to beselected not only based on the user's previous interaction data, butalso on a collective color preference of a group of users. Examples ofcollaborative filtering may include, but are not limited to, techniqueslike matrix factorization, single-value decomposition, neuralcollaborative filtering, and neural factorization machines.

At step 508, the content-prediction application identifies one or morecolors based on positions of the vectors in the embedding space. In someinstances, the color can be identified by calculating distances betweenvectors (e.g., Euclidean, Minkowski distance, Hamming distance, cosinedistance/similarity) and identifying a position in which the nearestdistance among the vectors, in a manner similar to techniquesimplemented in a Support Vector Machine. Graph coordinates of theidentified position can be converted into a set of pixel values thatcorrespond to the identified color. In some instances, another neuralnetwork model is applied to the graph coordinates of the identifiedposition to infer the set of pixel values that correspond to theidentified color.

Additionally or alternatively, the distance metric between vectors foridentifying color-palette preference may be determined by calculating anEarth Mover's Distance (EMD) value between the vectors. In thisimplementation, the neural network generates a plurality of vectors thatrepresent the web page, instead of a single vector representing the webpage as the above implementation. After the plurality of vectors foreach web page are plotted in the embedding space, the EMD between twovectors representing two web pages can be determined based on semanticdistances between words in the respective web pages, where words can beaccessed from an electronic lexical database called WordNet. Once thesemantic distances between words are obtained, the EMD calculates thesimilarity between two web pages with a many-to-many matching betweenwords. For example, three words in one web page are equivalent inmeaning to just one word in the other web page. The EMD that is lessthan a predetermined threshold can be formed into different clusters.Afterwards, each cluster is processed to identify a color that canassigned as a color-matching preference.

At step 510, the content-prediction application determines acolor-palette preference of the user based on the identified colors. Atstep 512, the content-prediction application selects one or more colorpalettes based on the color-palette preference. Similar toimplementations described in FIG. 4 , the content-prediction applicationcan identify a color of the color-palette preference and convert thecolor into a database query. The content-prediction application cansubmit the query to a database in the server system (e.g., thecolor-palette content 114 of FIG. 1 ) to retrieve a set of colorpalettes that include the color specified in the database query. Process500 terminates thereafter.

Predicting Content Items Using Convolutional Neural Network withEmbedding Layers

A content-prediction application (e.g., the content-predictionapplication 115 of FIG. 1 ) may receive the color palette, which caneither be selected by a user or predicted based on previous interactiondata. The content-prediction application may process the color paletteto predict one or more content items. A content item can include one ormore colors that match the color(s) specified in the color palette. Thecontent item may be identified by comparing a degree of similaritybetween color-palette vectors that represent one or more colors of thecolor palette and content vectors that represent the content items. Oncethe content items are identified, a catalog that includes the identifiedcontent items can be generated. In some instances, user-interfaceelements of a client application can be relocated such that theuser-interface elements associated with the content items are positionedat a visible portion of a user interface.

(a) Conversion of Color Palettes into Color-Palette Vectors of theColor-Embedding Space

In some embodiments, the content-prediction application generates acolor-palette vector based on a set of pixel values that represent acolor of the color palettes. In addition, the content-predictionapplication may continue to generate color-palette vectors for each ofthe other colors of the color palettes, in which the color may beassociated with same or different color-palette categories. As such, thecolor palette can be represented by a set of color-palette vectors thatcan be identified in the color-embedding space. For example, thecontent-prediction application selects a color associated with a maincolor-palette category or a neutral color-palette category of the colorpalette. The color can be a portion of the image that represents thecolor palette. In some instances, the color is identified based onmetadata associated with the color palette. The color can be processedto identify the set of pixel values. If the color is identified based onthe portion of the image, the content-prediction application may selecta pixel within the portion at a particular row and column and identifythe pixel values corresponding to the pixel. In some instances, sets ofpixel values are identified for each color in the color palette.

The content-prediction application may generate a color-palette vectorbased on the set of pixel values, in which the color-palette vectorrepresents the color at a first location within a color-embedding space.The color-palette vector may include a set of graph coordinates foridentifying the location of the color-palette vector at thecolor-embedding space. Two color-palette vectors having similar graphcoordinate values in the color-embedding space may indicate that ahigher degree of similarity relative to other vectors in thecolor-embedding space. In some instances, the content-predictionapplication applies a set of parameters (e.g., weights) to the generatedcolor-palette vector based on a type of color-palette category. Acontent item can be predicted not only based on its color thatcorresponds to one or more colors of the color-palette, but also on thetype of color-palette category that is associated with the one or morecolors of the color-palette.

Additionally or alternatively, a word-embedding neural network isapplied to one or more word tokens describing the color to generate thecolor-palette vector, instead of extracting the set of pixel values. Forexample, the one or more word tokens identify a word “red” or multiplewords such as “greenish blue.” Different types of word-embedding neuralnetwork can be used to generate the color-palette vector from the wordtoken, which may include neural-net language models (NNLM), globalvectors for word representation (GloVe), deep contextualized wordrepresentations (ELMo), and Word2vec, or any other machine-learningtechniques trained and configured to process word tokens to generatecolor-palette vectors.

(b) Prediction of Color Signatures of Content Items Based onConvolutional Neural Network Models

FIG. 6 illustrates a schematic diagram 600 for using convolutionalneural networks to predict color signatures of content items inaccordance with some embodiments. For illustrative purposes, the processimplemented in the schematic diagram 600 is described with reference tothe components illustrated in FIG. 1 , though other implementations arepossible. For example, the program code for content-predictionapplication 115 of FIG. 1 , which is stored in a non-transitorycomputer-readable medium, is executed by one or more processing devicesto cause the server system 102 to perform one or more operationsdescribed herein.

A content-item image 605 can be provided to the content-predictionapplication configured to apply one or more layers of a deepconvolutional neural network (DCNN) model. The content-item image 605may be structured as one or more arrays or matrices of pixel values. Insome instances, pixel value corresponding to a given pixel position isassociated with (for example) a general intensity value and/or anintensity value as it pertains to each of one or more colors (e.g., RGBvalues).

The content-prediction application can perform a series of operations onthe pixel values corresponding to the content-item image 605, which forma single convolutional layer for several types of DCNN architectures:(1) convolution; (2) non-linearity conversion (e.g., ReLU); (3) poolingor sub sampling; and (4) classification (e.g., Fully Connected Layer).In some instances, convolution operation of the content-predictionapplication preserves the spatial relationship between pixels bylearning image features using small squares of input data. For example,the content-item image 605 is considered as a matrix of pixel values, inwhich each pixel area of the matrix can be assigned with values. Inaddition, the content-item image 605 can be a black-and-white imagehaving pixel values ranging from 0 and 1. The content-item image 605 mayalternatively or additionally include color images that have threeassigned RGB values ranging from 0 and 255.

After the content-item image 605 is accessed, the content-predictionapplication may generate a feature map 610 based on the content-itemimage 605. In some embodiments, the feature extractor includes one ormore kernels or filters including a matrix of values that can “slideover” the matrix corresponding to the content-item image 605 so as togenerate the feature map 610. For example, a kernel is configured todetect edges, sharpen the image, or identify sets of pixels thatrepresent the subject in order to preprocess the data and providedetails that are relevant for image analysis of the classifier. In someinstances, a set of hyperparameters that correspond to the featureextraction are predefined (e.g., based on manual input).Feature-extraction hyperparameters may identify (for example) a stridefor each filter (e.g., 1-step, 2-step), a padding size, a kernel size,and/or a kernel shape.

Continuing with the example of FIG. 6 , the content-predictionapplication may perform a non-linearity (e.g., ReLU) conversion whichreplaces all negative pixel values in the feature map 610 by zero. TheReLU operation of the content-prediction application may introducenon-linearity in the DCNN model, as the convolution operation generatesa linear function output from the matrix values representative of thecontent-item image 605. The non-linearity conversion may additionally oralternatively include using sigmoid function and/or tan h function.

After the ReLU operation, the content-prediction application can performa pooling operation on the feature map 610 in order to reduce thespatial size of the feature map 610 and subsequently generate a pooledfeature map 615. In some embodiments, pooling reduces the dimensionalityof each feature map generated by the kernel of the feature extractor butretains the image-defining features (e.g., edges of an image) capturedin the content-item image 605. In some instances, the content-predictionapplication performs a max pooling operation to access an area withinthe feature map (e.g., 5×5 matrix within the feature map) and selectsthe element associated with the highest value. This operation can beiterated to traverse the entirety of the feature map 610, at which themax pooling operation completes the generation of the pooled feature map615. The content-prediction application may alternatively oradditionally perform an average pooling operation in place of the maxpooling operation which selects the sum or average value of the elementscaptured in the area within the feature map 610. By performing thepooling operations, the content-prediction application may achieveseveral technical advantages including capability of generating an inputrepresentation of the content-item image 605 that allows reduction ofnumber of parameters and computations within the CNN model, as wellscaling of invariant represent of the content-item image 605.

Continuing with the example of FIG. 6 , the content-predictionapplication may continue to apply one or more additional convolutionallayers on the content-item image 605 at which convolution, ReLU, andpooling operations can be performed on the pooled feature map 615. Insome instances, the second convolution layer generates a feature map620, a pooled feature map 625, and so forth. These operations arerepeated until the extracted features become equivariant to the scaleand translation of the fully connected layer.

After applying several convolutional layers, the content-predictionapplication may apply a fully connected layer 630, which may be amulti-layer perceptron network incorporating a softmax activationfunction or other types of linear or non-linear functions at an outputlayer 635. In some embodiments, the content-prediction application usesthe fully connected layer that accesses the extracted features andgenerates an output that includes a color signature that corresponds tothe content-item image 605. In some instances, the color signature is avector that includes an array of values that specify visualcharacteristics of the content-item image 605. Thereafter, the outputand the labels of the training dataset can be used as input for lossfunctions which process them to optimize the parameters in the DCNNmodel. For instance, the content-item image 605 can be classified by thecontent-prediction application to predict its color signature, such thata content vector generated based on the color signature can be comparedwith the color-palette vector to identify a degree of similarity. Insome instances, the content-prediction application additionally predictswhether the content item of the content-item image 605 should beassociated with a color-palette category, such as a main color-palettecategory, a neutral color-palette category, and an accent color-palettecategory. Values of the content vector generated from the colorsignature can be modified by applying one or more weight parameters, inwhich a value of the weight parameter is dependent of the type ofcolor-palette category predicted for the particular content item. Anyerror value from the output layer 635 (e.g., output data specifyingwrong color signature) can be calculated based on comparing the valuesfrom the output layer 635 and the labels in the training data. The errorvalue can then be used in backpropagation algorithms to adjust theparameters in the DCNN model and thus improve the accuracy of subsequentoutputs determined by the classifier.

It will be appreciated that, while FIG. 6 depicts using twoconvolutional layers to process the content-item image 605, differentnumbers of convolutional layers may be used (e.g., which may have aneffect of repeating these operations can be repeated by thecontent-prediction application one or more times). For example, outputsare determined by applying 10 or more convolutional layers to extractfeatures from the content-item image 605. In some embodiments, poolingoperations are omitted for one or more convolutional layers applied bythe classifier. In some embodiments, different versions of the DCNNmodel architecture are used by the classifier, including but not limitedto AlexNet, ZFNet, GoogLeNet, VGGNet, ResNets, DenseNet, etc.

In some instances, the content-item image 605 is processed by thecontent-prediction application without applying the fully connectedlayer 630 of the DCNN model. As such, the output from the layers of theDCNN model can be used as the color signature of the content-item image605. The color signature can be used to generate a content vector thatcan represent the content-item image 605. The content vector may includegraph coordinates for identifying a location (e.g., locationcoordinates) of the content item in the color-embedding space. In someinstances, an embedding layer is applied to the content vector to reducethe dimensionality of the embedding vector. For example, the contentvector representative of the color signature of the content item isassociated with values having 20 dimensions, at which the embeddinglayer can reduce the dimensions of the values to 10. In some instances,multiple content vectors are generated for the content-item image 605that includes multiple colors (e.g., a checkered pattern includingbeige, brown, black colors).

(c) Predicting Content Based on Color-Embedding Space

Once the color signature is generated for the content-item image 605,the content vector of the content-item image 605 can be plotted in thecolor-embedding space, at which the embedding vector is compared withcolor-palette vectors in the color-embedding space. The content vectorsof a plurality of content-item images can be compared with each of thecolor-palette vectors to generate a set of distance metrics, that isindicative of a degree of similarity between the colors of thecontent-item image 605 and a given color of the color palette. Thedistance metric can be a Euclidean distance calculated between theembedding vectors.

In some embodiments, the distance metric between two vectors includesother types of metrics including, but are not limited to, Minkowskidistance, Hamming distance, and cosine distance/similarity. For example,the Euclidean distance d(A, B) is calculated for a color-palette vectorrepresentative of the color palette A=[a₁, a₂, . . . , a_(n)] and acontent vector representative of a content item of the catalog B=[b₁,b₂, . . . b_(n)] A lower Euclidean distance value may indicate a higherdegree of similarity between vectors A and B.

d(A,B)=√{square root over ((a ₁ −b ₁)²+(a ₂ −b ₂)²+ . . . +(a _(n) −b_(n))²)}

In another example, the cosine similarity θ(A, B) is calculated for acolor-palette vector representative of the color palette A=[a₁, a₂, . .. , a_(n)] and a content vector representative of an item of the catalogB=[b₁, b₂, . . . b_(n)]. A higher cosine value may indicate a higherdegree of similarity between vectors A and B.

${\theta\left( {A,B} \right)} = \frac{A*B}{{A}*{B}}$

In some instances, each vector having a distance metric lower than apredetermined threshold is identified, and the item corresponding to thevector is selected to be displayed on the user interface. If thedistance metric corresponds to a cosine similarity, the content item canbe selected if the distance metric exceeds the predetermined threshold.Additionally or alternatively, the set of items can be ranked based ontheir respective distance metrics, from the lowest distance metric valueto the highest distance metric value. The catalog can be generated suchthat the set of items are presented in accordance with the determinedrankings.

If the content item is represented by set of content vectors, thecontent-prediction application calculates a distance metric (e.g., theEuclidean distance) between each of the set of content vectors and itsnearest color-palette vector in the color embedding space, therebygenerating a set of distance metrics. An average or a median value ofthe set of distance metrics can be calculated, which is then identifiedwhether such value is less than the predetermined threshold. If so, thecontent-prediction application can select the content item (e.g., a sofawith checkered pattern having red, pink, and white colors) representedby the average or median value of the set of distance metrics.

Example of a Computing Environment

Any suitable computing system or group of computing systems can be usedfor performing the operations described herein. For example, FIG. 7depicts a computing system 700 that can implement any of the computingsystems or environments discussed above. In some embodiments, thecomputing system 700 includes a processing device 702 that executes thecontent-prediction application 115 including the content-predictionapplication 115, a memory that stores various data computed or used bythe content-prediction application 115, an input device 714 (e.g., amouse, a stylus, a touchpad, a touchscreen, etc.), and an output device716 that presents output to a user (e.g., a display device that displaysgraphical content generated by the content-prediction application 115).For illustrative purposes, FIG. 7 depicts a single computing system onwhich the content-prediction application 115 is executed, and the inputdevice 714 and output device 716 are present. But these applications,datasets, and devices can be stored or included across differentcomputing systems having devices similar to the devices depicted in FIG.7 .

The example of FIG. 7 includes a processing device 702 communicativelycoupled to one or more memory devices 704. The processing device 702executes computer-executable program code stored in a memory device 704,accesses information stored in the memory device 704, or both. Examplesof the processing device 702 include a microprocessor, anapplication-specific integrated circuit (“ASIC”), a field-programmablegate array (“FPGA”), or any other suitable processing device. Theprocessing device 702 can include any number of processing devices,including a single processing device.

The memory device 704 includes any suitable non-transitorycomputer-readable medium for storing data, program code, or both. Acomputer-readable medium can include any electronic, optical, magnetic,or other storage device capable of providing a processor withcomputer-readable instructions or other program code. Non-limitingexamples of a computer-readable medium include a magnetic disk, a memorychip, a ROM, a RAM, an ASIC, optical storage, magnetic tape or othermagnetic storage, or any other medium from which a processing device canread instructions. The instructions may include processor-specificinstructions generated by a compiler or an interpreter from code writtenin any suitable computer-programming language, including, for example,C, C++, C#, Visual Basic, Java, Python, Perl, JavaScript, andActionScript.

The computing system 700 may also include a number of external orinternal devices, such as a display device 710, or other input or outputdevices. For example, the computing system 700 is shown with one or moreinput/output (“I/O”) interfaces 708. An I/O interface 708 can receiveinput from input devices or provide output to output devices. One ormore buses 706 are also included in the computing system 700. Each bus706 communicatively couples one or more components of the computingsystem 700 to each other or to an external component.

The computing system 700 executes program code that configures theprocessing device 702 to perform one or more of the operations describedherein. The program code includes, for example, code implementing thecontent-prediction application 115 or other suitable applications thatperform one or more operations described herein. The program code may beresident in the memory device 704 or any suitable computer-readablemedium and may be executed by the processing device 702 or any othersuitable processor. In some embodiments, all modules in thecontent-prediction application 115 are stored in the memory device 704,as depicted in FIG. 7 . In additional or alternative embodiments, one ormore of these modules from the content-prediction application 115 arestored in different memory devices of different computing systems.

In some embodiments, the computing system 700 also includes a networkinterface device 712. The network interface device 712 includes anydevice or group of devices suitable for establishing a wired or wirelessdata connection to one or more data networks. Non-limiting examples ofthe network interface device 712 include an Ethernet network adapter, amodem, and/or the like. The computing system 700 is able to communicatewith one or more other computing devices (e.g., a computing device thatreceives inputs for content-prediction application 115 or displaysoutputs of the content-prediction application 115) via a data networkusing the network interface device 712.

An input device 714 can include any device or group of devices suitablefor receiving visual, auditory, or other suitable input that controls oraffects the operations of the processing device 702. Non-limitingexamples of the input device 714 include a touchscreen, stylus, a mouse,a keyboard, a microphone, a separate mobile computing device, etc. Anoutput device 716 can include any device or group of devices suitablefor providing visual, auditory, or other suitable sensory output.Non-limiting examples of the output device 716 include a touchscreen, amonitor, a separate mobile computing device, etc.

Although FIG. 7 depicts the input device 714 and the output device 716as being local to the computing device that executes thecontent-prediction application 115, other implementations are possible.For instance, in some embodiments, one or more of the input device 714and the output device 716 include a remote client-computing device thatcommunicates with the computing system 700 via the network interfacedevice 712 using one or more data networks described herein.

General Considerations

Numerous specific details are set forth herein to provide a thoroughunderstanding of the claimed subject matter. However, those skilled inthe art will understand that the claimed subject matter may be practicedwithout these specific details. In other instances, methods,apparatuses, or systems that would be known by one of ordinary skillhave not been described in detail so as not to obscure claimed subjectmatter.

Unless specifically stated otherwise, it is appreciated that throughoutthis specification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining,” and “identifying” or the likerefer to actions or processes of a computing device, such as one or morecomputers or a similar electronic computing device or devices, thatmanipulate or transform data represented as physical electronic ormagnetic quantities within memories, registers, or other informationstorage devices, transmission devices, or display devices of thecomputing platform.

The system or systems discussed herein are not limited to any particularhardware architecture or configuration. A computing device can includeany suitable arrangement of components that provide a result conditionedon one or more inputs. Suitable computing devices include multi-purposemicroprocessor-based computer systems accessing stored software thatprograms or configures the computing system from a general purposecomputing apparatus to a specialized computing apparatus implementingone or more embodiments of the present subject matter. Any suitableprogramming, scripting, or other type of language or combinations oflanguages may be used to implement the teachings contained herein insoftware to be used in programming or configuring a computing device.

Embodiments of the methods disclosed herein may be performed in theoperation of such computing devices. The order of the blocks presentedin the examples above can be varied—for example, blocks can bere-ordered, combined, and/or broken into sub-blocks. Certain blocks orprocesses can be performed in parallel.

The use of “adapted to” or “configured to” herein is meant as open andinclusive language that does not foreclose devices adapted to orconfigured to perform additional tasks or steps. Additionally, the useof “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Headings, lists, and numbering includedherein are for ease of explanation only and are not meant to belimiting.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing, may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude the inclusion of suchmodifications, variations, and/or additions to the present subjectmatter as would be readily apparent to one of ordinary skill in the art.

What is claimed is:
 1. A method comprising: accessing, by a content-prediction application, a set of web pages accessed by a user via a browser application, wherein a first web page of the set of web pages includes a first layout that substantially matches a second layout of a second web page of the set of web pages; applying, by the content-prediction application, a machine-learning model to encode the set of web pages into a set of vectors, wherein each vector of the set of vectors represents a corresponding web page of the set of web pages, wherein the first web page is encoded into a first vector of the set of vectors, and wherein the second web page is encoded into a second vector of the set of vectors; associating, by the content-prediction application, the set of vectors with an embedding space, wherein the first vector is associated with the embedding space such that a position of the first vector in the embedding space is in proximity to a position of the second vector in the embedding space; identifying, by the content-prediction application, colors based at least in part on the positions of the first vector and the second vector in the embedding space; determining, by the content-prediction application, a color-palette preference of the user based on the identified colors; selecting, by the content-prediction application, a color palette based on the color-palette preference, wherein the color palette includes a set of colors, and wherein the set of colors includes the identified colors and additional colors from two or more color-palette categories; and generating, by the content-prediction application, an item catalog that includes an item having a color that matches at least one color of the set of colors of the color palette.
 2. The method of claim 1, wherein accessing the set of web pages accessed by the user includes: parsing, by the content-prediction application, the first web page to identify a set of web page elements, wherein each web page element of the set of web page elements includes an identifier and location information that specifies a location of the web page element relative to the first web page; and determining, by the content-prediction application, the first layout of the first web page.
 3. The method of claim 1, wherein associating the set of vectors in an embedding space includes associating the set of vectors with an additional set of vectors, wherein each vector of the additional set of vectors represents an additional web page accessed by another user.
 4. The method of claim 1, wherein identifying colors based at least in part on the positions of the first vector and the second vector in the embedding space includes: determining, by the content-prediction application, a distance between the position of the first vector and the position of the second vector; identifying, by the content-prediction application, a nearest position between the first vector and the second vector based on the determined distance; converting, by the content-prediction application, graph coordinates of the nearest position to a set of pixel values; and identifying, by the content-prediction application, a particular color of the identified colors based on the set of pixel values.
 5. The method of claim 1, further comprising converting, by the content-prediction application, the color palette into a database query, wherein the database query is used to determine the item of the item catalog.
 6. The method of claim 1, wherein the machine-learning model is an artificial neural network.
 7. The method of claim 1, wherein the two or more color-palette categories includes: (i) a main color-palette category that includes a primary color of the color palette; and (ii) a neutral color-palette category that includes a complementary color that complements the primary color of the color palette.
 8. A system comprising: one or more processors; and a memory having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to perform operations comprising: accessing a set of web pages accessed by a user via a browser application, wherein a first web page of the set of web pages includes a first layout that substantially matches a second layout of a second web page of the set of web pages; applying a machine-learning model to encode the set of web pages into a set of vectors, wherein each vector of the set of vectors represents a corresponding web page of the set of web pages, wherein the first web page is encoded into a first vector of the set of vectors, and wherein the second web page is encoded into a second vector of the set of vectors; associating the set of vectors with an embedding space, wherein the first vector is associated with the embedding space such that a position of the first vector in the embedding space is in proximity to a position of the second vector in the embedding space; identifying colors based at least in part on the positions of the first vector and the second vector in the embedding space; determining a color-palette preference of the user based on the identified colors; selecting a color palette based on the color-palette preference, wherein the color palette includes a set of colors, and wherein the set of colors includes the identified colors and additional colors from two or more color-palette categories; and generating an item catalog that includes an item having a color that matches at least one color of the set of colors of the color palette.
 9. The system of claim 8, wherein accessing the set of web pages accessed by the user includes: parsing the first web page to identify a set of web page elements, wherein each web page element of the set of web page elements includes an identifier and location information that specifies a location of the web page element relative to the first web page; and determining the first layout of the first web page.
 10. The system of claim 8, wherein associating the set of vectors in an embedding space includes associating the set of vectors with an additional set of vectors, wherein each vector of the additional set of vectors represents an additional web page accessed by another user.
 11. The system of claim 8, wherein identifying colors based at least in part on the positions of the first vector and the second vector in the embedding space includes: determining a distance between the position of the first vector and the position of the second vector; identifying a nearest position between the first vector and the second vector based on the determined distance; converting graph coordinates of the nearest position to a set of pixel values; and identifying a particular color of the identified colors based on the set of pixel values.
 12. The system of claim 8, wherein the instructions further cause the one or more processors to perform operations comprising: converting the color palette into a database query, wherein the database query is used to determine the item of the item catalog.
 13. The system of claim 8, wherein the machine-learning model is an artificial neural network.
 14. The system of claim 8, wherein the two or more color-palette categories includes: (i) a main color-palette category that includes a primary color of the color palette; and (ii) a neutral color-palette category that includes a complementary color that complements the primary color of the color palette.
 15. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform actions including: accessing a set of web pages accessed by a user via a browser application, wherein a first web page of the set of web pages includes a first layout that substantially matches a second layout of a second web page of the set of web pages; applying a machine-learning model to encode the set of web pages into a set of vectors, wherein each vector of the set of vectors represents a corresponding web page of the set of web pages, wherein the first web page is encoded into a first vector of the set of vectors, and wherein the second web page is encoded into a second vector of the set of vectors; associating the set of vectors with an embedding space, wherein the first vector is associated with the embedding space such that a position of the first vector in the embedding space is in proximity to a position of the second vector in the embedding space; identifying colors based at least in part on the positions of the first vector and the second vector in the embedding space; determining a color-palette preference of the user based on the identified colors; selecting a color palette based on the color-palette preference, wherein the color palette includes a set of colors, and wherein the set of colors includes the identified colors and additional colors from two or more color-palette categories; and generating an item catalog that includes an item having a color that matches at least one color of the set of colors of the color palette.
 16. The computer program product of claim 15, wherein accessing the set of web pages accessed by the user includes: parsing the first web page to identify a set of web page elements, wherein each web page element of the set of web page elements includes an identifier and location information that specifies a location of the web page element relative to the first web page; and determining the first layout of the first web page.
 17. The computer program product of claim 15, wherein associating the set of vectors in an embedding space includes associating the set of vectors with an additional set of vectors, wherein each vector of the additional set of vectors represents an additional web page accessed by another user.
 18. The computer program product of claim 15, wherein identifying colors based at least in part on the positions of the first vector and the second vector in the embedding space includes: determining a distance between the position of the first vector and the position of the second vector; identifying a nearest position between the first vector and the second vector based on the determined distance; converting graph coordinates of the nearest position to a set of pixel values; and identifying a particular color of the identified colors based on the set of pixel values.
 19. The computer program product of claim 15, wherein the instructions further cause the one or more processors to perform operations comprising: converting the color palette into a database query, wherein the database query is used to determine the item of the item catalog.
 20. The computer program product of claim 15, wherein the two or more color-palette categories includes: (i) a main color-palette category that includes a primary color of the color palette; and (ii) a neutral color-palette category that includes a complementary color that complements the primary color of the color palette. 